Gaming and other electronic interactive environments allow for a wide variety activities between multiple users, as well as interactivity between a single user and the environment. While many gaming or virtual environments allow for entertainment, content generation is an additional benefit.
In current gaming or virtual environments, there is a pronounced limitation in the ability to generate audio and video content. The limitation relates to the environment itself and the ability of the user to control an avatar or otherwise interact with the environment.
Existing techniques for audio and video content generation are based on set, pre-defined parameters and the virtual user having a direct engagement with the content generator. By way of example, one type of content generator may be a virtual musical instrument, the audio content generated when the user virtually touches or otherwise engages the instrument, such as selecting keys of a keyboard. Another musical example may be a predefined set of controller buttons toggled to existing note-playing elements of an instrument, such as selecting the depression of one or more strings on a guitar. In these systems, the content generation is very static and solely controlled by a direct interaction with the user to the instrument itself.
Existing techniques focus on gesture control by the user and/or re-sequencing of pre-composed musical units. These techniques merely seek to emulate real-world musical performances (e.g. seeking expressiveness via emulating the gestures a musician would make if playing an acoustic instrument) or by making sure the musical instrument is always coherent to the context by providing, typically tempo synched, musical fragments that the player has some limited agency in re-arranging through gameplay.
The existing content generation is thus inherently limited by the source content available in the virtual space, e.g. which musical instruments and/or composed musical pieces have been pre-programmed into the space. None of the existing content generation techniques provide for user-generated purely dynamic content. As such, there exists a need for processing operations modifying audio and/or video content in a virtual environment, providing for output generation via one or more output devices.